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Background / Context: 

Experimental and quasi-experimental designs play a central role in estimating cause-effect 
relationships in education, psychology, and many other fields of the social and behavioral 
sciences. In implementing the designs for causal inference, researchers frequently rely on 
Campbell’s tradition of ruling out threats to internal validity like selection, maturation, history, 
or instrumentation effects (Shadish, Cook & Campbell, 2002). Though the conceptualization of 
internal validity and its threats is useful from a practical point of view, a more formal 
representation of the designs helps in precisely defining the causal estimands of interest and in 
discussing the assumptions required for identifying the causal estimands. The Rubin Causal 
Model (RCM) with its potential outcomes frameworks (Holland, 1986) offers a very convenient 
way of defining causal estimands and deriving identification assumptions like the stable-unit- 
treatment-value assumption (SUTVA; Rubin, 1990), strong ignorability for propensity score 
matching designs (Rosenbaum & Rubin, 1983), or the monotonicity assumption for instrumental 
variable designs (Angrist, Imbens, & Rubin, 1996). 

However, RCM is not the only way to formalize causal inference. Recently, graph-theoretical 
models that exploit substantive knowledge about the data- generating mechanism have become 
more popular for causal investigations in artificial intelligence, epidemiology or sociology 
(Morgan & Winship, 2012; Pearl, 2009; Spirtes, Glymour & Schemes, 2000). The graphical 
approach to causal inference is rather new to the social sciences like education and psychology, 
but an increasing number of researchers find graphical models appealing because they naturally 
convey substantive theories to causal graphs without relying on pure logical and mathematical 
expressions. Moreover, causal graphs also allow researchers to represent features of the study 
designs and data collection process that affect the identification of causal effects. This paper 
discusses the causal graphs for experimental and quasi-experimental designs and thereby 
provides a new perspective on the designs’ mechanics and assumptions. 

Purpose / Objective / Research Question / Focus of Study: 

We investigate graphical models for experimental and quasi-experimental designs (Steiner et al, 
in press) and focus on the nonparametric identification of the causal effects (that is, without 
making any functional form or distributional assumptions). In particular, we present and discuss 
the causal graphs for randomized controlled trials (RCTs), regression discontinuity (RD) designs, 
instrumental variable (IV) designs, and matching and propensity score (PS) designs. For each 
design we show how the assumptions required for identifying a causal effect are encoded in the 
graphical representation. We believe that causal graphs make the assumptions more transparent 
and easier to understand for applied researchers. The causal graphs also reveal new insights 
about the quasi-experimental designs. For instance, the propensity score is a collider variable that 
offsets confounding bias via collider bias. Or, the conditional causal graphs for compliers, 
always takers and never takers of the IV design make it very clear why the causal effect is only 
identified for compliers. 
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Significance / Novelty of study: 

In drawing causal graphs for different (quasi- Experimental designs, we introduce novel concepts 
and explanations of the designs’ causal identifying mechanisms. For the RD design, we 
introduce the concept of a limiting graph at the cutoff score which directly explains why the 
average treatment effect at the cutoff score is nonparametrically identified while the treatment 
effects to the left and right of the cutoff score are not identifiable (without further assumptions). 
For the IV design, we provide a simple but formally precise graphical explanation of why the 
average treatment effect is identified only for compliers but not for always takers and never 
takers. From the IV graphs one can also directly see how the complier average treatment effect is 
computed from observed data consisted of the mixture population of those latent subgroups. 
Finally, for PS designs we demonstrate that the PS is a collider variable that offsets the 
confounding bias via collider bias. In addition to these novel formulations of quasi-experimental 
designs, a comparison of the designs’ causal graphs impressively reveals that stronger designs 
(like RCTs or RD designs) results in simpler graphs due to researchers’ increasing control over 
the assignment mechanism. 

Usefulness / Applicability of Method: 

The graphical representation of (quasi-)experiments helps practitioners in getting a better 
understanding of the designs’ assumptions and limitations and, finally, in doing better causal 
studies. We also think that the causal graphs we present are a valuable tool for teaching (quasi)- 
experimental designs. 

Findings / Results: 

Figure 1(1) shows a typical data generating process for an observational study, where the Xs 
represent the confounders that prevent a direct identification of the treatment’s (Z) causal effect 
on the outcome (F). Figure 1(2) shows a stylized graph of Figure 1(1), where X represents the set 
of confounders X\ to X] while the other independent variables became a part of the independent 
error terms which are no longer shown. If not all confounders X are reliably measured or if a 
credible instrumental variable cannot be identified, then the average treatment effect is not 
identifiable from observational data. Thus, one might first think about an RCT or RD design in 
order to identify an average treatment effect. 

The causal graph for a RCT is shown in Figure 2, where R denotes randomization. The 
assumptions encoded by missing arrows in the graph are the following: i) there is no arrow R —> 
Y because randomization must not have a direct effect on Y under SUTVA; ii) there is no arrow 
X — > Z because of perfect compliance and no attrition; iii) there is no arrow R —+X because X 
should be measured before treatment assignment (i.e., pre-treatment variables). Under those 
assumptions, we see the graph satisfies the back-door criterion since there is no path between Z 
and Y that contains an arrow into Z and thus the causal effect is nonparametrically identified 
(Pearl, 2009). Conversely, if any of those assumptions is not met (that is, we need to draw at 
least one of the arrows R — > F, X — > Z, or R —*■ X), then the corresponding back-door paths 
between Z and Y make the causal identification infeasible. 
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Figure 3(1) shows the causal graph for a sharp RD design, where A is the continuous assignment 
variable. Under the absence of noncompliance, the causal graph encodes the assumption that A 
exclusively determines the treatment status Z, that is, there is no arrow X — > Z. However, the 
graph also implies that the assignment by itself could affect the outcome (A — > Y) or be 
correlated with other potential confounders (A <-> X). Without limiting identification to around 
of cutoff score, the two back-door paths of i) Z <— A — > Y, ii) Z 4 — A <-> X — > Y (and the lack 
of overlap) prevent the causal identification on the effect of Z on Y. However, identification 
becomes possible if one investigates the casual effect only in an infinitesimally close 
neighborhood around the cutoff as represented by the limiting graph at the cutoff score in Figure 
3(2). The limiting graph satisfies the back-door criterion and the causal effect is identified. Note 
that the limiting graph is essentially equivalent to the causal graph of an RCT implying that RD 
designs are actually RCTs at the cutoff score. 

The causal graph of an IV design is depicted in Figure 4. An IV is a variable that affects the 
outcome Y only via treatment Z but has otherwise no effect on Y. Thus, the graph shows the 
arrow IV —> Z but no other arrows between IV and other variables (exclusion restriction 
assumption). The strategy IV designs employ to circumvent the confounding due to X is 
identifying two effects of TV on Z and TV on Y rather than trying to directly identify the effect of 
Z on Y. Since the effect of TV on Y (average IV effect; AIVE) is not affected by X (graphically, 
the path TV — > Z <— X — > Y is naturally blocked due to the collider Z on this path), one can 
calculate the effect of Z on Y as the ratio between AIVE (IV —*■ Z — > Y) and the effect of TV on Z 
(IV — > Z). Under the monotonicity or no-defier assumption the treatment effect is identified for 
the latent subpopulation of compliers (CATE; Angrist, Imbens & Rubin, 1996). To see the 
mechanics behind the identification of CATEs, we introduce conditional graphs in Figure 5 for 
latent subgroups of compliers (1) and always takers & never takers (2). By definition, compliers 
strictly follow the IV’ s assignment, therefore, there is no arrow X — > Z, while always takers and 
never takers are never influenced by the IV, thus, there is no arrow TV — > Z. For the complier 
graph, the effect IV —> Z is necessarily equal to 1 (because Z = IV) and thus the AIVE is identical 
to CATE, t c , in Figure 6(1). Thus, if the latent subpopulation of compliers would be identifiable, 

CATE would be directly identified (while the effect for always and never takers cannot be 
identified without conditioning on X). But since we rarely know who the compliers (they are 
latent) are we only can use the data of compliers, always and never takers together (no defiers) as 
presented in Figure 5(3). From the no defiers graph, however, the CATE become identifiable 
under monotonicity assumption, because i) the AIVE (TV Z-* Y) is the product of the 
complier probability and CATE, that is, yr c , ii) the average effect TV — > Z is given by the 

complier probability y- P(Si = Complier ), and finally iii) taking the ratio of the two, the effect of 

yx 

Z on Y is computed by — — = r c , which is the CATE. The identification of IV designs can be 

Y 

directly applied to noncompliance issue in RCTs and RD designs (i.e., fuzzy RD designs) as in 
Figure 7 and 8, respectively. 

Finally, Figure 9 and 10 show the causal graphs for matching and PS designs, respectively. In 
those designs researchers typically are not in control of the selection to treatment. However, if 
the strong ignorability assumption is met (Rosenbaum & Rubin, 1983), then by conditioning on a 
matching indicator or the PS the causal effects can be identified. The crucial step in in matching 
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or PS designs is the computation of the matching indicator or the PS. Since the matching 
indicator is determined from both the treatment Z and the confounders X the indicator S 
necessarily is a collider on the path between Z and X (Mansournia, Hernan & Greenland, 2013; 
Shahar & Shahar, 2012; Figure 9(2)). Then conditioning on S creates a collider bias but it exactly 
offsets the confounding path X — > Z (Figure 9(3)) and thus achieves X J_ Z I S . Since the 
resulting graph of the matched data in Figure 9(4) now resembles the graph of an RCT, the 
causal effect is identified without further conditioning. We argue that the very same mechanics 
apply to all PS designs in general, that is, regardless of the choice of a specific analytic PS 
method — matching, stratification, or weighting. Again, since the PS is computed from both Z 
and X, it is a collider on the path between Z and X (Figure 10(2)) and creates collider bias once 
one conditions on it via matching, stratifying or weighting. The collider bias exactly offsets the 
confounding path (Figure 10(3)) and finally establishes the independence X _L Z I PS in Figure 
10(4). 

Conclusions: 

This paper presents the causal graphs of experimental and quasi-experimental designs. For quasi- 
experimental designs we demonstrate that a causal treatment effect can be identified if we can 
isolate a subpopulation whose graph resembles the graph of an RCT. For RD designs the 
corresponding subpopulation is given by the population in the very close neighborhood around 
the cutoff score; For the IV design it is the subpopulation of compliers; For matching designs it 
is the matched population and for PS stratification or weighting designs it is the stratified or 
weighted population. The causal graphs also show that the identification of causal effects rests 
on stronger assumptions as the researchers’ control over the study diminishes. More control 
usually implies simpler the data- generating mechanism and, thus, relative ease of identification. 
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Appendix B. Tables and Figures 


Figure 1. Causal graph of an observational study: complete and stylized representation. 
(1) Complete graph (2) Stylized graph 
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Figure 2. Data generating graph of a randomized controlled trial. 
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Figure 3. Causal graphs of a regression discontinuity design: (1) data generating graph and (2) 
limiting graph for A = a c +£' with £■— » 0. 

(1) Data generating graph (2) Limiting graph 
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Figure 4. Data generating graph of an instrumental variable design. 
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Figure 5. Data generating IV graphs for (1) compliers (C), (2) always takers and never takers 
(A+N), and (3) compliers, always takers and never takers together (i.e., no defiers). 
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(3) No defiers (C+A+N) 
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Figure 6. IV graphs for (1) compliers and (2) compliers, always takers and never takers together 
(i.e., no defiers). 

(1) Compliers (C) (2) No defiers (C+A+N) 
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Figure 7. Causal graph of the randomized controlled trial with noncompliance. 
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Figure 8. Causal graphs for the fuzzy regression discontinuity design: (1) data generating graph 
and (2) limiting graph for ^~ a c- £ w ith £■— » (). 

(1) Data generating graph (2) Limiting graph 
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Figure 9. Causal graphs for a matching design (with constant matching ratio): (1) Data 
generating DAG, (2) computation of match indicator S, (3) conditioning on the match indicator S, 
and (4) the independence structure after matching (S = 1). 

(1) Data generating graph (2) Computation of match indicator 
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(3) Conditioning on the match indicator (4) Independence structure after matching (S = 1) 

X X 

✓ 



SREE Spring 2016 Conference Abstract Template 


B-4 



Figure 10. Causal graphs for propensity score designs: (1) Data generating DAG, (2) 
computation of the propensity score (PS), (3) conditioning on the PS, and (4) the independence 
structure after PS-adjustment. 

(1) Data generating graph (2) Computation of PS 
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(3) Conditioning on the PS (4) Independence structure after PS adjustment 
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